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The rise and growing prevalence of juvenile delinquency is a matter of 
concern for many parties. This study aims to establish a research instrument 
in the form of a questionnaire that can be deployed to assess the learning 
environment perceived by high school students. This research endeavor 
constitutes a developmental study, wherein the outcomes are a single survey 


instrument encompassing six variables, nineteen indicators, and forty 
questions. The data-collecting process involved the utilization of a Google 
Keywords: Form across five schools in five districts, containing a total of 1615 
participants. The analysis of expert data was conducted utilizing V. Aiken and 
field trials employing confirmatory factor analysis (CFA) Second Order. The 
findings of this study indicate that the diagnostic survey instrument used to 
assess the learning environment's impact on the mental health of high school 
Mental health students demonstrated validity, as evidenced by loading factor values 
Psychometry exceeding the established minimal threshold. The reliability of the instrument 
remains insufficient. This survey can be utilized to detect adolescent persistent 
tendencies carried out by students or other school members that interfere with 
mental health: the emergence and significant raising of juvenile delinquency. 


Diagnostic survey 
Instrument 
Learning environment 


This is an open access article under the CC BY-SA license. 


Corresponding Author: 


Risky Setiawan 

Educational Research and Evaluation, Graduated School, Universitas Negeri Yogyakarta 
Colombo Street No. 1 Depok Sleman, D.I. Yogyakarta, Indonesia 

Email: riskysetiawan @uny.ac.id 


1. INTRODUCTION 

The rise of crime, juvenile delinquency, and drug abuse in the school environment is a government 
concern, especially the Ministry of Education and Culture and Technology in overcoming it. One of the leading 
causes is an unsafe and comfortable school environment. The learning environment is one of the leading 
indicators that support student learning success in school [1], [2]. If the learning environment is not good, it 
will impact students’ mental health [3], [4]. Research in the United States estimates that annually, 20-25% of 
children and adolescents experience mental health problems, and 40% of them meet diagnostic criteria for 
various types of mental disorders, not including children and adolescents who are at risk and have not been 
diagnosed but whose conditions affect daily functioning and wellbeing [5], [6]. Epidemiological research in 
the United States shows that 1 in 10 children show symptoms of depression before the age of 14, and 20% of 
children aged 16-17 experience anxiety, mood, behavioral disorders, and acute mental disorders [7], [8]. 

Most mental disorders begin in adolescence and early adulthood (10 to 24), and poor mental health is 
associated with adverse educational, health, and social outcomes [9], [10]. School is said to be a context for 
the positive promotion and prevention of mental health problems. Mental health includes mental, emotional, 
and spiritual [11]. Students with good mental health are characterized as building and developing resilience in 
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the face of stresses in life [12]. This resilience must be developed through family life and the school 
environment [13]. So, creating a positive school environment is urgently needed. 

Causes of unhealthy student mentality at school include acts of bullying, sexual violence, corporal 
punishment, and even drug abuse. Some juvenile delinquency has reached criminal cases, such as murder by 
piracy, which is rife in Yogyakarta [14]. It tremendously impacts students who are victims of mental and 
psychological hazards. A sense of security and comfort for students at school will be able to optimize their 
mental health well. Students with good mental health will give birth to psychologically prosperous individuals 
to achieve national education goals and produce quality and outstanding students. 

The main problems that cause trauma disorders to psychosocial post-traumatic stress disorder cause 
various social issues. Identification as early as possible can be made with the innovative construction of survey 
instruments that assessment centers in developed countries are now developing. A good survey instrument can 
prevent mental hazards in schools [15]. In addition, this learning environment survey is a diagnostic assessment 
used to determine student learning readiness. The follow-up survey results developed are knowing the mapping 
of students' abilities and whether there are indications of mental and psychological hazards in schools [16], 
[17]. Once identified through these instruments, schools can take action by providing guidance and counseling 
services to foster students, for example, information services, content mastery services, group guidance 
services, and individual counseling services [18], [19]. Thus, the products developed can increase preventive 
efforts in keeping the student learning environment safe and conducive to learning. 

Mental health in schools is crucial for students and staff, encompassing emotional balance, stress 
coping, healthy interpersonal relationships, and effective learning. The American Psychological Association 
(APA) and World Health Organization (WHO) emphasize the importance of mental health in schools, promoting 
access to services and security and removing stigma. The National Alliance on Mental Illness (NAMD 
underscores the need for staff education and support. Educational psychologists play a vital role in understanding 
and supporting mental health, providing counseling, training, and strategies for creating a supportive environment. 
Educators and counselors play a critical role in fostering a supportive learning environment. 

The literature indicates a high prevalence of mental health issues among senior high school students 
[20], [21]. In aggregate, the publications suggest that mental health issues pose a substantial concern among 
the population of senior high school students, hence warranting increased focus on their mental well-being 
several risk variables are associated with mental health problems among senior high school students. In a study 
conducted by Kim [20], it was seen that females exhibited lifestyle-related behaviors that were less conducive 
to good health and were more susceptible to experiencing adverse mental health outcomes. The findings above 
indicate that it is imperative to devise and execute treatments targeting several domains, such as educational 
institutions, familial units, social networks, and the individual adolescent, to mitigate the occurrence of mental 
health problems among senior high school students. 

Some experiences triggering mental health issues by senior high school students based on literature 
consist of some variables. The first is bullying. Bullying is a pervasive concern inside educational institutions, 
impacting students across various age groups, genders, and ethnic backgrounds. Oliveira et al. [22] reveal that 
the predominant factor contributing to bullying incidents in Brazilian schools was related to physical 
attractiveness. Zych et al. [23] opines that bullying is a multifaceted psychosocial phenomenon characterized 
by a hierarchical power dynamic and a culture of silence. Conversely, the presence of positive relationships 
with both instructors and classmates, as well as a perceived sense of academic achievement, were identified as 
characteristics that offer protection against such behaviors. The scholarly articles indicate that bullying is 
complex, necessitating a comprehensive school-wide intervention strategy for prevention and resolution. 
Consequently, it is recommended that anti-bullying initiatives incorporate cognitive-motivational aspects in 
their design and implementation. 

The scholarly literature posits that manifestations of intolerance inside educational institutions 
encompass various phenomena, such as racism, xenophobia, and religious intolerance. According to Juwita et 
al. [24] study, there is evidence to suggest that male pupils enrolled in a religious-based elementary school 
exhibit a higher degree of ignorance towards differences, while their female counterparts demonstrate a greater 
sensitivity towards diversity. It must be addressed that this mindset will fully grow as time goes on and age 
adds. It has to be anticipated by implementing some ways at home and school. In general, the scholarly articles 
indicate that intolerance inside educational institutions is a multifaceted matter necessitating a comprehensive 
strategy for effective resolution. A preventive way to detect intolerance at schools is by spreading the diagnostic 
intolerance questionnaire toward the experiences that students directly experienced or witnessed. 

Sexual violence is a pervasive concern within educational institutions, wherein distinct manifestations 
of such abuse transpire across various contexts. According to the research of Pereira et al. [25], the lack of 
educational environments addressing the issue of sexual violence is a contributing cause to its prevalence 
within educational institutions. The implementation of efficient measures aimed at avoiding sexual violence 
within educational institutions is a matter of great importance. However, further research is required to 
ascertain the optimal tactics that yield the highest efficacy. According to the study conducted by Lester [26], 
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many programs, such as cognitive-behavioral therapies, social-emotional interventions, and peer mentorship 
and mediation programs, can be implemented to reduce the acts of sexual violence at schools. However, further 
investigation is required to ascertain the enduring impacts of these interventions and identify the essential 
components of effective interventions. As a solution, the author suggests including sexual education in the 
regular school curriculum, including conducting a diagnostic test for all parties of schools, with the 
participation of families. 

Drug abuse has adverse short- and long-term effects on students' mental health. Mental health that is 
damaged by drug abuse are mental disorders [27], physical and emotional damage [28], symptoms of anxiety 
and posttraumatic stress [29], and depressive symptoms [30]. Not only that, drug abuse in schools can have 
adverse effects on academic performance. Zemba [31] found that drug abuse led to poor academic performance, 
absenteeism, and low concentration in class among pupils. Teresia [32] also highlights the adverse effects of 
drug abuse in schools, including poor academic performance. Pereira et al. [33] found that implementing drug 
abuse prevention programs in schools was associated with the manager's experience in education and with the 
teaching strategies of the school. Handrianto et al. [34] emphasize the importance of teachers in drug abuse 
prevention in schools, as students spend most of their quality time at school, and teachers' roles have resulted 
in their being the critical factor in the prevention of drug consumption. 

Gender equality lately tends to be an essential issue in all fields of business, including education. In 
the educational field, gender equality can be achieved by realizing gender parities. The promotion of gender 
equality within educational institutions can be achieved by employing gender-responsive learning strategies, 
incorporating gender considerations into lesson planning, organizing classrooms in a manner that is sensitive 
to gender dynamics, utilizing inclusive terminology, and fostering gender-inclusive classroom interactions 
[35]. Educators must make much effort to promote gender equality in the school environment, including in the 
classroom during teaching and learning activities. 

Physical punishment is associated with the development of mental health disorders. Duran and Ensom 
apply Eriksonian theory to explain how physical punishment may disrupt the successful resolution of 
psychosocial crises in childhood and adolescence, leading to mental health problems [36]. Three research 
studies from Afifi et al. [37]-[39] suggest that physical punishment is harmful to psychological and physical 
health and should be avoided. 

After several exposures, an instrument is needed to formulate the following problem: i) What is the 
learning environment diagnostic survey (LEDS) constructed to improve mental health in senior high school? 
ii) What is the validity and reliability of the LEDS constructed to improve mental health in senior high school? 


2. METHOD 

This study is a development research utilizing instrument development procedures. This research aims 
to explain the methods of instrument development and prove the quality of the instruments to measure the 
learning environment that supports mental health in senior secondary schools in a province in Indonesia. The 
development procedures conducted in this research are determining objectives, collecting supported theories, 
designing a blueprint of the instrument based on related theories and research, composing the indicators and items 
of the instrument, validating the instrument (content validity and construct validity), and finalizing the instrument. 


2.1. Participants 

This study engaged 1615 students of senior secondary school from 5 regencies in Indonesia. Because 
of the size of the population and the researcher's inability to draw a representative sample of it at random, 
convenience sampling was used in this study [40]-[43]. The participants’ grades are not restricted to gaining 
more comprehensive data from new senior secondary school students, first graders, or third graders. The 
participants' demography is clearly illustrated in Table 1. 


Table 1. The demographic data of the participants 


Category Frequency __ Percentage 

Regency 

Sleman 415 26 
Bantul 170 10 
Gunung Kidul 373 23 
Kota Yogyakarta 405 25 
Kulonprogo 252 16 
Grade 

X 566 35 
XI 569 35 
XII 480 30 


J Edu & Learn, Vol. 18, No. 3, August 2024: 690-698 


J Edu & Learn ISSN: 2089-9823 o 693 
2.2. Data collection 

The data collection is done through a survey. Participants from five regencies were invited to fill out 
the questionnaire spread from the WhatsApp application in the form of Google Forms. The collection of the 


data was in a classroom accompanied by the teachers. 


2.3. Data analysis 

In analyzing the data gained from content validity, V Aiken was utilized. The range score from 1-5 
given by the experts to each item of the instruments was calculated. For construct validity, the result of tryouts 
from Google Forms was analyzed using SmartPLS to test the validity and estimate the reliability. The quality 
of the instrument's items was calculated using item response theory assisted by jMetrik. 


3. RESULTS AND DISCUSSION 

After the instrument's construction is done in composing, the instrument's quality is tested by proving 
the validity and reliability as well as the item quality of the instruments. Validity can be divided into three 
categories: criterion validity, content validity, and construct validity [44]. All sources of validity, however, are 
optional to be fulfilled or confirmed for validity. According to Lissitz and Samuelson [45], an analysis of the 
test's content and an empirical analysis of the test results on the test instrument can reveal information about a 
test device's reliability. Therefore, construct and content validity are used to examine the validity of this 
instrument. The ability of an instrument to measure the content or scope of the material as intended at the outset 
is referred to as the instrument's content validity. Correct values and appropriate sampling methods are two 
crucial components of content validity [46]. Focus group discussions (FGDs) can be used to test the content 
validity of an article [47]. 


3.1. Content validity 

The FGD was carried out in these stages: i) brainstorming, conveying an explanation from the chief 
of researchers about the concept; ii) presentation by a resource person from the FGD members; and 
iii) conclusions from the discussion results. The FGD consisted of eight people: two policyholders, two 
educational evaluation experts, two classroom learning experts, and two educational psychology experts. There 
were 71 instruments in the FGD consisting of 55 direct question (DQ) instruments and 16 situational judgment 
test (SJT) instruments. The results of the brainstorming showed that: i) it was necessary to reduce the items 
because they were deemed to be too many, at least to 50%; ii) the SJT instrument has a statement that is too 
long, so it must be summarized; and iii) change the instrument indicator from DQ to SJT because it is difficult 
to measure that point directly. The following are the results of the FGD, which will be analyzed using the V- 
Aiken method to prove the validity of the instrument content. From Table 2, it can be concluded that all items 
of the instruments are valued highly by the experts. It is assumed that the instrument is ready to be tested in the 
next step: to test the construct validity. 


Table 2. The results of content validity by eight experts 


Instrument Indicator V-Aiken Instrument Indicator V Aiken 
Bullying Physical bullying 0.96875 Sexual violence Rape 0.9375 
Bullying Physical bullying 0.90625 Sexual violence Rape 0.875 
Bullying Verbal bullying 0.84375 Sexual violence Rape 0.9375 
Bullying Verbal bullying 0.84375 Sexual violence Sexual intimidation 0.96875 
Bullying Social bullying 0.96875 Sexual violence Sexual intimidation 0.9375 
Bullying Social bullying 0.96875 Sexual violence Sexual intimidation 0.90625 
Bullying Cyberbullying 1 Sexual violence Sexual harassment 1 
Bullying Cyberbullying 1 Sexual violence Sexual harassment 0.96875 
Bullying Sexual bullying 0.96875 Sexual violence Sexual harassment 1 
Bullying Sexual bullying 1 Drug abuse Drug use 0.84375 
Intolerance Religious intolerance 0.96875 Drug abuse Drug use 0.71875 
Intolerance Religious intolerance 0.9375 Drug abuse Drug trafficking 0.78125 
Intolerance Religious intolerance 0.96875 Drug abuse Drug trafficking 0.90625 
Intolerance Economic intolerance 0.96875 Gender equality Equality of treatment in learning 0.96875 
Intolerance Economic intolerance 0.9375 Gender equality Educational equality 0.96875 
Intolerance Economic intolerance 0.96875 Gender equality Equality of treatment outside the classroom 1 
Intolerance Cultural intolerance 0.9375 Gender equality Right to obey the same rules 0.96875 
Intolerance Cultural intolerance 0.96875 Physical punishment Light punishment 0.9375 
Intolerance Cultural intolerance 0.96875 Physical punishment Light punishment 0.9375 
Physical punishment Severe punishment 0.9375 
Physical punishment Severe punishment 0.96875 
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3.2. Construct validity 

The construct validity is done through a tryout. The tryout was carried out at five high schools in five 
districts in Yogyakarta Province. The sample obtained was a total of 1615 students. Data were analyzed using 
confirmatory factor analysis (CFA) in second order. Using the help of the SmartPLS application in analyzing 
trial data of a learning environment diagnostic survey instrument to determine the mental health quality of 1615 
high school students in grades X, XI, and XII in Yogyakarta City and Regency, the construct of a learning 
environment diagnostic survey instrument was obtained. The construct of the learning environment diagnostic 
survey instrument consists of 6 variables, namely the variables of bullying (bullying), intolerance, sexual 
violence, drug abuse, gender equality, and corporal punishment. Each variable has an indicator that explains 
these variables. The bullying variable has five indicators: physical, verbal, social, cyberbullying, and sexual. 
Intolerance has three indicators: religious intolerance, economic intolerance, and cultural intolerance. 

Meanwhile, sexual violence is represented by three indicators, namely rape, sexual intimidation, and 
sexual harassment. Next, the drug abuse variable is defined by two indicators, namely drug use and distribution. 
The gender equality variable is defined by four indicators: equality of treatment in learning, equality of 
education, equality of treatment outside the classroom, and the right to obey the same rules. The last is the 
physical punishment variable, which has two indicators, namely indicators of light punishment and severe 
punishment. The learning environment diagnostic survey instrument consists of 6 variables, which are reduced 
to 19 indicators, 40 questions, and statements are formulated as DQ), 28 items, and a SJT with 12 items. The 
analysis results show quality in the form of validity and reliability of the learning environment diagnostic 
survey instrument, learning environment diagnostic survey instrument variables, and loading factors for each 
learning environment diagnostic survey instrument item. From the data analysis process, the composite 
reliability (CR) value is obtained, which can determine internal consistency reliability, with a standard CR 
value > 0.7 (adequate internal consistency) [48]. Other values obtained from SmartPLS analysis in Cronbach's 
Alpha should be > 0.7. 


3.3. Validity 

The validity standard is met by looking at the internal consistency reliability analysis value using CR, 
which meets the standard. Internal consistency reliability answers whether the statement items used in the 
questionnaire can measure the construct to be studied [48]. Nunnally [44] determined the internal consistency 
Reliability value with CR values ranging between 0.6 and 0.7 to be acceptable for exploratory studies. Another 
way to interpret an instrument construct is with Cronbach's Alpha. The alpha coefficient that should be obtained 
on an excellent predicate is > 0.7 [44]. 

Analysis of pretesting results uses convergent validity to see how well an item can measure the same 
construct in a study [48]. Convergent validity can be analyzed by calculating three tests: factor loading (factor 
loading/outer loading), CR, and average variance extracted (AVE). The first is an assessment of external 
loading or what is usually called a loading factor. Table 3 shows the loading factors from the analysis process 
using smartPLS. 

Table 3 shows the analysis results of each indicator on the learning environment diagnostic instrument 
variable. According to Kamis et al. [49], the loading factor must be greater than 0.7, and CR must be greater 
than 0.5. Of the 40 loading factors from the 40 items of the learning environment diagnostic survey instrument, 
31 had a loading factor of > 0.7, 7 had a loading factor value of > 0.7, and even two items had a minus score. 
Factor loading values in the range of 0.4 to 0.7 should be considered for removal because if small loading 
factors are removed, it is likely that the AVE value will increase. 

However, several theories differentiate the lowest factor loading value. Mardapi [47] states that the 
minor factor loading value that can still be mentioned by finger is the loading factor with a value of 0.3, which 
is still acceptable. Thus, based on Mardapi [47], the 38 loading factors of the 40 construct items of the learning 
environment diagnostic survey instrument have met the valid standard, namely, they are more significant than 
0.3. Two items with a minus loading factor value must be considered to be eliminated or reviewed. These two 
items come from the variable sexual violence with indicators of sexual intimidation and sexual harassment. It 
could be because discussions related to this topic are still very taboo, so respondents' openness to this matter 
still needs to be improved. Apart from that, another possibility that causes this is the type of statement or 
question made into favorable and unfavorable forms. The two minus loading factors are good 
statements/questions on the sexual violence variable. It allows for ambiguity among respondents. 
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Table 3. Loading factors of each item in the instrument 


Indicator Cronbach's Alpha Rho _A CR AVE Items Outer loading 
Physical bullying 0.471 0.482 0.789 0.652 FSK1 0.764 
FSK2 0.849 
Verbal bullying 0.396 0.403 0.767 0.622 VBLI 0.830 
VBL2 0.745 
Social bullying 0.541 0.542 0.813 0.685 SSL1 0.818 
SSL2 0.837 
Cyberbullying 0.538 0.596 0.806 0.677 CBY1 0.742 
CBY2 0.896 
Sexual bullying 0.091 0.104 0.674 0.521 SKL1 0.864 
SKL2 0.545 
Religious intolerance 0.427 0.468 0.728 0.484 ITAL 0.468 
ITA2 0.787 
ITA3 0.783 
Economic intolerance 0.710 0.722 0.840 0.638 ITE] 0.845 
ITE2 0.858 
ITE3 0.682 
Cultural intolerance 0.499 0.500 0.750 0.502 ITB1 0.638 
ITB2 0.765 
ITB3 0.716 
Rape 0.638 0.645 0.807 0.584 PMK1 0.775 


PMK2 0.677 
PMK3 0.832 


Sexual intimidation -0.487 0.750 0.429 0.667 IDS1 -0.792 
IDS2 0.677 
IDS3 0.832 
Sexual harassment -0.777 0.727 0.353 0.644 PCS1 -0.820 
PCS2 0.841 
PCS3 0.743 
Drug use 0.261 0.265 0.729 0.574 PMN1 0.711 
PMN2 0.802 
Drug trafficking 0.163 0.163 0.705 0.544 PDN1 0.718 
PDN2 0.757 
Equality of treatment in learning 1.000 1.000 1.000 1.000 KPP 1.000 
Educational equality 1.000 1.000 1.000 1.000 KMP 1.000 
Equality of treatment outside the classroom 1.000 1.000 1.000 1.000 KPL 1.000 
Right to obey the same rules 1.000 1.000 1.000 1.000 HMP 1.000 
Light punishment 0.222 0.222 0.720 0.562 HKRI1 0.747 
HKR2 0.752 
Severe punishment 0.158 0.162 0.701 0.542 HKB1 0.804 


HKB2 0.661 


3.4. Reliability 

After obtaining conclusions regarding the construct validity of the learning environment diagnostic 
survey instrument, which is included in the valid category for most items, the reliability estimates need to be 
studied further to show the quality of the learning environment diagnostic survey instrument. The assessment 
regarding reliability estimates can be seen from several values presented in Table 4. The rules for determining 
whether the reliability of an instrument construct is good or not have been explained at the beginning. 


Table 4. Construct reliability estimation and learning environment instrument variables 


Instrument Variable Cronbach's Alpha _ Rho_A CR AVE 
Learning environment 0.380 0.869 0.310 0.154 
Bullying 0.709 0.736 0.792 0.287 

Intolerance 0.721 0.727 0.801 0.312 

Sexual violence 0.316 0.790 0.613 0.372 

Drug use 0.372 0.378 0.679 0.347 

Gender equality 0.221 0.230 0.629 0.301 

Physical punishment 0.284 0.290 0.649 0.318 


From Table 4, it can be concluded that overall, judging from the values of CR (should be > 0.7), 
Cronbach's Alpha (> 0.7), Rho_A (> 0.7), and AVE (> 0.5), the construct of a learning environment diagnostic 
survey instrument to assess the mental health of high school students does not yet have high reliability. The 
recorded Cronbcah's Alpha value was 0.380 and CR 0.310 (< 0.7), which is still very far from the standard. 
Even though it has a good Rho_A value, the AVE value also does not meet the standard > 0.5, namely only 
0.154. 
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Low-reliability estimates are also owned by several latent variables in the construct of the learning 
environment diagnostic survey instrument, where the variables of sexual violence, drug abuse, gender equality, 
and corporal punishment have Cornbcah's Alpha, CR, and AVE values, which are below standard (> 0.7 and 
> 0.5). Meanwhile, two variables, bullying and intolerance, have Cr values that meet the standard, namely 
> 0.7. However, it is not followed by Cornbach's Alpha and AVE values. Thus, it can be concluded that the 
learning environment diagnostic survey instrument has low reliability. Even though each item is valid and can 
measure each indicator studied, this instrument still needs to provide steady or reliable research results when 
used in different research settings. The implications of research on the development of learning environment 
surveys provide less than optimal results, especially in the reliability coefficient. The variety of research 
subjects indicates this. Of the five schools that were the main objects, it turned out that they could still not 
provide adequate results. However, the instrument will be better developed if integrated with broader learning 
environment indicators. 


4. CONCLUSION 

The highlight of this study was to test the quality of content, construct quality, validity and reliability 
of a questionnaire-based survey instrument to measure a learning environment that supports students' mental 
health. The learning environment tool to support student mental health has six variables: bullying, harassment, 
violence, drug abuse, gender equality, and corporal punishment. There are at least two points that support these 
six variables. The final variable is corporal punishment which is presented in two categories, namely light 
punishment and heavy punishment. The survey instruments are direct questionnaires and situation assessment 
tests. There are 28 DQ related to bullying, intolerance, and violence. For the remaining three variables, there 
are 12 items as decision tests. The instrument's validity is low because the CR, Cronbach's alpha and calculated 
variance are below normal values (> 0.7 and > 0.5). This tool can be used periodically at all academic levels 
to assess patterns that may negatively impact students' mental health. This situation makes responding to 
questions and comments difficult for students with different skill levels. 
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